Winner-Take-All Networks of O(N) Complexity

We have designed, fabricated, and tested a series of compact CMOS integrated circuits that realize the winner-take-all function. These analog, continuous-time circuits use only O(n) of interconnect to perform this function. We have also modified the winner-take-all circuit, realizing a circuit that computes local nonlinear inhibition. Two general types of inhibition mediate activity in neural systems: subtractive inhibition, which sets a zero level for the computation, and multiplicative (nonlinear) inhibition, which regulates the gain of the computation. We report a physical realization of general nonlinear inhibition in its extreme form, known as winner-take-all. We have designed and fabricated a series of compact, completely functional CMOS integrated circuits that realize the winner-take-all function, using the full analog nature of the medium. This circuit has been used successfully as a component in several VLSI sensory systems that perform auditory localization (Lazzaro and Mead, in press) and visual stereopsis (Mahowald and Delbruck, 1988). Winnertake-all circuits with over 170 inputs function correctly in these sensory systems. We have also modified this global winner-take-all circuit, realizing a circuit that computes local nonlinear inhibition. The circuit allows multiple winners in the network, and is well suited for use in systems that represent a feature space topographically and that process several features in parallel. We have designed, fabricated, and tested a CMOS integrated circuit that computes locally the winner-take-all function of spatially ordered input. THE WINNER-TAKE-ALL CIRCUIT Figure 1 is a schematic diagram of the winner-take-all circuit. A single wire, associated with the potential Vc, computes the inhibition for the entire circuit; for an n neuron circuit, this wire is O(n) long. To compute the global inhibition, each neuron k contributes a current onto this common wire, using transistor T2k . To apply this global inhibition locally, each neuron responds to the common wire voltage Vc, using transistor T1k . This computation is continuous in time; no clocks are used. The circuit exhibits no hysteresis, and operates with a time constant related to the size of the largest input. The output representation of the circuit is not binary; the winning output encodes the logarithm of its associated input. T11 T1n T21 T2n I1 In